Salatschleuder R3

Only for additional information:
I released this program for a coding-contest at www.planet-quellcodes.de in 2006 in german. Now I translated it to english for www.donationcoder.com in 2010.

Heres the last version of the Salatschleuder!

what has changed since the contest release?
- Fileerrors for multiselection are fixed
- the turbocharacterreplacer is now activated to replace several characters in only on step all together
- the structures are now neater and use less memory
- errors in the word-splitter-routine have been fixed. "test:" -> ':' was mistreated as a part of the word
- the hash-routine to search the words was improved and less words with the same charactercontent will be eliminated (better results)
- the hash-routine has been accelerated and the reading of dictionaries and generation of prefixes- & suffixes is now much faster
- during the creation of a dictionary the sorting can be deactivated. This results in a worse compression, but the results can get better if the direction of the words is decremental sorted (as examle a top 100-list has the most common word as first entry)
- the hash-values of the word variants is now dynamic, because the length is delimited to 32 bits. This means the bits are splitted dynamicalle by the size of the lexicon and the suffix/prefix-list! In the contest release the lexicon was limited to only 65535 words and bigger dictionaries with a few million words)
- optimization has shrinken down the program a little bit
- reading a new text for translation removes now the previous text correctly (in former releases this led to crashes)
- compatibility down to Windows 95

The File Ergebnisse.xls contains different tests

Please be restictive with prefix-suffix-combinations. A big dictionary with only less p/s can melt down your memory very fast. My experiance is the recognition rate much hiher and uses less memory by including more words in the dictionary and only includes very large suffix list and only very less prefixes. If you have an empty p/s list you must read a file with no entires!

Even if you have the original text you want to translate completely in the lexicon you will not be able to get more than 98 percent translation rate. This means: 1-2% of all words are words with the same length and starting/ending letters but different letters inside.

Here the old readme:

The name of the program is Salatschleuder
Programming-language: C++ (MS Visual Studio)
Compatibility: Win 95 upwards

Foren-Nick: Crush
EMail: Crush@Bigfoot.de

Description:

In the upper part of the windows are the function buttions (declaration at the end)
The upper text window is the originaltext you want to shuffle
The lower text window is the shufflewindow that containts the shuffled text from the upper one or the translations

If an specific text is read and a shuffled text will be translated theres an additional check how many characters from the original text with position is suiting with the translated text.

Buttons:

Simpliest usage:
Theyre sorted that you can first decide to set the switches on the right belonging to the functions on their left side.
After this create or load a created dictionary and perhaps prefixes/suffixes.
Then the left row of buttons can be clicked from the top to the bottom to get forward the right way.

"Load" reads the original text

"Save saves" the shuffled or translated text from the lower text window

"Shuffle" shuffles the original text from the upper window to to lower shuffle window

"Translate" translates the text from the shuffle window

"Load Shuffletext" loads a saved shuffled text in the shuffle window

"Create dictionary" transforms a textfile to a dictionary for translation. At first you have to select the textfile and then 
the save name of the dictionary.

"Read dictionary" loads a dictionary and creates in deep search mode all word variants and the builts up the needed hashtables for translation.

All switches are beneath the buttons (functions) that uses them. So you have to turn the switches before clicking on the lefthanded buttons.

The switch "Deep search" creates a word list with additional variants of the dictionary while reading it. It must be active before you read the lexcon(s) and the prefix/suffix-tables and of course during translation!This function needs a lot of memory and can take a very long time for the creation but the translation of the texts is due to the hashmaps very fast (the complete bible can be translated in about 10-15 seconds on a normal PC). If your memory is getting too small you have to reduce the suffix and prefix files a little bit. If you use an empty file as a suffix list only prefix variants will be created (thats much less in amount). Otherwise from each word will be for each suffix variant all prefix variants and the memory usage grows quadratic.

The switch "Sort the words" turns on/off the sorting of the words while creation of a dictionary. Turning it off off will result in a worse compression but the recognition rate can get better if the words have been used in a decreasing line like a Top-100 list that has the most used words at first. The reason words like "brat" or "bart"
 cannot be differed from the program. The first word that has the same letters and the same start/ending characters will be included into the dictionary. The other words with the same attributes will be discarded.

have a lot of fun with it

Crush 2010

I used the follow links to find wordlists:
http://wordlist.sourceforge.net/
http://www.e-medtools.com/openmedspel.html
http://pocketcm.com/phorum/file.php?14,file=825,filename=en_wiktionary.zip
http://www.outpost9.com/files/WordLists.html
http://www.bckelk.ukfsn.org/menu.html#Wordlists
http://icon.shef.ac.uk/Moby/
ftp://nic.funet.fi/pub/unix/security/dictionaries/DEC-collection/
http://www.catb.org/jargon/download.html
http://aspell.net/
ftp://ftp.gnu.org/gnu/aspell/dict/0index.html
http://www.gutenberg.org/etext/3201
ftp://ftp.ox.ac.uk/pub/wordlists/Moby/
ftp://ftp.itri.bton.ac.uk/bnc/
http://www.kilgarriff.co.uk/bnc-readme.html
http://ucrel.lancs.ac.uk/bncfreq/flists.html
http://en.wiktionary.org/wiki/Wiktionary:Frequency_lists
http://www.cis.doshisha.ac.jp/kkitao/library/resource/corpus/corpus.htm
http://personal.cityu.edu.hk/~davidlee/devotedtocorpora/CBLLinks.htm
http://helmer.aksis.uib.no/corpora/2001-1/0447.html
http://wortschatz.uni-leipzig.de/html/wliste.html
ftp://ftp.ox.ac.uk/pub/wordlists/
ftp://ftp.fu-berlin.de/pub/unix/security/dictionaries/
http://garbo.uwasa.fi/pc/spell.html
http://www.cotse.com/tools/wordlists1.htm
http://www.cotse.com/tools/wordlists2.htm
http://www.net-comber.com/wordurls.html
http://www.crosswordman.com/wordlist.html
http://www.puzzlers.org/dokuwiki/doku.php
http://stardict.sourceforge.net/Dictionaries_dictd-www.dict.org.php
http://fmg-www.cs.ucla.edu/geoff/ispell-dictionaries.html
http://www.gutenberg.org/browse/authors/w#a1132
http://en.wiktionary.org/wiki/Wiktionary:Main_Page
http://www.alt-usage-english.org/
http://simple.wiktionary.org/wiki/Wiktionary:BNC_spoken_freq
http://simple.wiktionary.org/wiki/Wiktionary:Academic_word_list
http://grammar.about.com/od/words/a/comsuffixes.htm
http://www.prefixsuffix.com/affixes.php?navblks=1011000
http://www.learnenglish.de/grammar/prefixtext.htm
http://www.learnenglish.de/grammar/suffixtext.htm
